Add sort integration benchmark #13306
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
alamb
approved these changes
Nov 13, 2024
There was a problem hiding this comment.
Thank you @2010YOUY01 -- I am sorry for the delay in reviewing
I found this PR really well commented and documented. ❤️
What would you think about adding this benchmark to the dfbench benchmark binary rather than creating an entirely new one?
Perhaps it can be SortIntegration or SortTPCH for a name?
Each different binary has substantial overhead when building (and it is a different target to build, etc) so reducing the total number of binaries I think helps the overall CI time
| /// - Thin variant: `l_partkey` column with `BIGINT` type (1 column) | ||
| /// - Wide variant: all columns except for possible key columns (12 columns) | ||
| const SORT_QUERIES: [&'static str; 10] = [ | ||
| // Q1: 1 sort key (type: INTEGER, cardinality: 7) + 1 payload column |
There was a problem hiding this comment.
thank you for your annotations here. 👍
|
@alamb Thank you, this makes sense to me. Updated. |
2010YOUY01
force-pushed
the
sort-bench
branch
from
affe136
to
349ba55
Compare
|
Thank you very much @2010YOUY01 -- very cool |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #.
Rationale for this change
I noticed there is no benchmark to test sorting the whole relational table: existing sort benchmark is only for a single
SortExec, and this can't test how would end-to-end large sort query scale to multiple CPU cores. With integration test, it is possible to see the combined performance of local sort on small batches and final step multi way sort-preserving merge.The benchmark includes 10 queries to sort the entire
lineitemtable in TPCH dataset, with different characteristics. For example: with different number of sort key/ payload columns, different sort key types and cardinality, etc. Also it is easy to add more benchmark queries.More details see
sort_integration.rsWhat changes are included in this PR?
Added a single benchmark binary for benchmark. It can be executed with:
# Under benchmarks/ ./bench.sh run sort_integrationQuery run results comparing sorting
lineitemtable with scaling factor 1 and 5Are these changes tested?
Are there any user-facing changes?